The present invention relates to a device and method of retrieving high-speed motion. More particularly, the present invention relates to a high-speed vector retrieval device that can obtain at high speed an optimum motion vector to improve the compression efficiency in a moving picture encoding device that compressive-encodes moving images.
The inter-motion-compensation-frame encoding is a procedure of retrieving the pixel value Xlm closest to Xij; from signals in the (k-l) frame to predict the pixel Xijk in k frame. Particularly, the above encoding means the procedure of searching for a block with highest correlation in block units divided every constant rectangular region. Spacial shifts |i-l| and |j-m| between Xij and Xlm are transmitted as additional information (motion vector).
Conventionally, a method of calculating a region with a highest correlation and a difference to reduce data amount to be encoded is exemplified as an effective means of improving the image quality in an image compression device that performs inter-frame-predictive encoding.
According to such a method, more widening the vector retrieval range allows accurate retrieval but the processing amount increases sharply. When a high-speed compression process is required, the operation amount is generally reduced by narrowing the retrieval range or by stopping the retrieval when correlation with a certain high level is found. This method is disclosed in JP-A No. 271514/1998 (first prior art).
However, this method has the disadvantage in that the image quality is deteriorated because the possibility is that the retrieval process is interrupted regardless of the existence of higher correlation becomes strong.
As another previous example, JP-A No. 32969/1996 (second prior art) discloses the technique of dynamically changing the retrieval range using the correlation to the same location as that in a previous frame.
In the above prior-art example, since the magnitude of movement of an image is not reflected to the retrieval range, even a region with small motion may be often retrieved broadly, so that the processing time is not always shortened. For example, in motion retrieval of a moved boundary portion of an object, the retrieval may be performed over a broader range because of low correlation to the same location in the previous frame.
As further another prior-art example, JP-A No. 191352/1998 (third prior art) discloses the technique of performing in parallel the correlation seeking process. However, this method has the disadvantage in that plural arithmetic units are required, thus resulting in an increased circuit scale. For example, this method requires plural CPUs to realize with a personal computer.
In summary, the above three prior arts have the following disadvantages.
In the first prior art, the retrieval process may be interrupted in spite of the existence of a portion with high correlation so that the image quality in degraded.
In the second prior art, because an image with large motion is not reflected to the retrieval range, even a portion with small movement may be often retrieved over a broader range, so that the processing time is not always shortened. For example, in motion retrieval of a moved boundary portion of an object, the retrieval may be performed over a broader range because of low correlation to the same location in the previous frame.
In the third prior art, plural arithmetic units are required, thus resulting in an increased circuit scale. For example, plural CPUs are required to realize with a personal computer.
The present invention is made to overcome the above-mentioned problems.
The objective of the present invention is to provide a novel motion vector retrieval device that can detect unerring motion vectors at high speed.
Another objective of the present invention is to provide a method of retrieving unerring motion vectors at high speed.
According to the present invention, a high-speed motion retrieval device comprises an compressive encoding device; the compression encoding device including image input means for receiving an image in one frame units; motion retrieval means for obtaining a correlation between a current frame and a previous frame output from the image input means; inter-block differentiating means for performing a differential operation to a block at a location corresponding to a motion vector on a reference frame, the motion vector being a vector to the location where a minimum predictive error has been obtained through retrieval of the motion retrieval means; frequency conversion means for converting block differential data obtained by the inter-block differentiating means into a frequency component; quantization means for quantizing the frequency component; and variable-length encoding means for compressively encoding a quantized output from the quantization means; the motion vector retrieval means including first-stage motion retrieval means and second-stage motion retrieval means; wherein the first-stage motion retrieval means retrieves a retrieval range to be decided by referring to a retrieval result in the previous frame, ends a motion retrieval process at the time the range has been retrieved if a predictive error is smaller than a threshold value, retrieves a region not retrieved by the first-stage motion retrieval means within a retrieval range predetermined by the second-stage motion retrieval means if the predictive error is larger than the threshold value, thus obtaining a region with a minimum predictive error, and wherein the first stage motion retrieval means performs retrieval in two steps and dynamically varies a first-stage retrieval range, thus performing motion vector retrieval with less erroneous detection at high speed.
In the high-speed retrieval device, the first-stage motion retrieval means retrieves a predetermined retrieval range without a special process since when image data is input to the image input means, it is decided whether or not retrieval information data can be referred to but the information cannot be retrieved at initial retrieval.
In the high-speed retrieval device, the threshold value comprises a retrieval result of a macroblock at the same location as that on the previous frame.
In the high-speed retrieval device, a retrieval range of the first-stage motion retrieval means is a square region of which each side has a length twice the motion vector length obtained in the previous frame.
In the high-speed retrieval device, when retrieval information can be referred to, the steps are performed of capturing information about motion vector or predictive error of a macroblock at the same location as that in said previous frame, predicting a location with a minimum predictive error by performing retrieval based on the motion vector information of said previous frame, with a retrieval start location shifted by the vector, retrieving the location with particular emphasis, and deciding a first-stage motion retrieval range based on captured vector information.
In the high-speed retrieval device, the retrieval range is a maximum or median value of a vector of an ambient block already retrieved within a frame, or a maximum or median value of each vector at the same location on past several frames, and wherein if the minimum value of a predictive error obtained over the retrieval range is smaller than a predictive error in a block at the same location on a previous frame, a retrieval process is ended, or if not, a retrieval process is continued.
The high-speed retrieval device further comprises retrieval information storage means inserted between the input of the first-stage motion retrieval means and the output of the second-stage motion retrieval means, the retrieval information storage means storing a motion vector value or a minimum predictive error value, obtained by the first-stage motion retrieval means and the second motion retrieval means, for next frame motion retrieval.
The high-speed retrieval device further comprises motion compensation means connected to the first-stage motion retrieval means and the second-stage motion retrieval means. The motion compensation means inverse-quantizes and inverse-frequency converts an output of said quantization means to convert a frequency component into an image component and then creates data about reference frame from the image component.
Referring FIG. 1, the first-stage motion retrieval means 12 retrieves a retrieval range determined by referring to the retrieval result in the previous frame. If a minimum predictive error is smaller than a threshold value, the motion retrieval process is ended at the time the retrieval range has been retrieved. If the minimum predictive error is larger than the threshold value, the second-stage motion retrieval means 13 retrieves the region which has not been retrieved through the first-stage motion retrieval within a predetermined retrieval range, thus obtaining the location with a minimum predictive error.